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Letter from the Editor 

Nuance. It's a simple word that does not adequately seem to reflect the complexity it represents. 

In reflecting upon the three articles presented in this volume, it is the first word that came 
to mind. The work of selecting peers, defining discounting, and discussing expectations all 

Association for 

require attention to nuance, and these articles suggest important reasons why this is the case. institutional Research 

As our world becomes ever-more informed by data, there is an ability to better understand the 
complexity of issues, yet at the same time there exists a belief that more data will make issues easier to understand and 
explain in simple formats. 

The Obama Administration's focus on college scorecards, rankings, and shopping sheets to guide the college selection 
process — and the metrics that comprise these efforts — serve as examples of a perception of simplicity. Yet as D'Allegro 
and Zhou point out, selecting peers at the institutional level requires complex analysis. One would think that students' 
selection processes would consist of more exploration than simply clicking on a criteria or two as well. This assumes, 
of course, that the data elements have common definitions and are operationalized the same way. However, that 
assumption is not accurate, as highlighted by Davis and Redd and by Seifert, Wells, Saunders, and Gopaul. 

I am reminded that appropriate use of data requires an appreciation for nuance. I believe that after reading the fine work 
presented in these three papers you will regain your appreciation for it as well. 



Sincerely, 



Christopher M. Mullin 
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Abstract 

Peer selection based on the similarity 
of a couple of institutional parameters, 
by itself, is insufficient. Several other 
considerations, including clarity of 
purpose, alignment of institutional 
information to that purpose, 
identification of appropriate statistical 
procedures, review of preliminary peer 
sets, and the application of additional 
metrics need to be part of the process. 

At the heart of the paper is a detailed 
description of a mixed-methods 
approach deployed to identify 
institutional peer and aspirant groups 
for a private nonprofit 4-year liberal arts 
college. As part of the methodology, 
an aspirant index is proposed and 
explained by the authors. This 
coefficient is applied to a preliminary 
set of institutions to further refine the 
aspirant list. 

This paper inventories the methods 
documented in other research and 


resources that can be used to select 
peers. This compendium is intended 
to inform customized amalgamation 
of methods that could potentially 
augment future peer selection 
endeavors and benchmarking studies. 

INTRODUCTION 

Peer comparisons have become 
increasingly common (Gater, 2003; 
Huxley, 2009; McLaughlin, Howard, 

& McLaughlin, 201 1;Trainer, 2008). 
Comparative analyses address the 
demand for accountability, provide 
benchmark targets, justify budget and 
planning decisions, and complement 
competitor appraisals (McLaughlin 
& Howard, 2005). Accordingly, 
comparisons with other institutions 
seem to be gaining legitimacy (Eckles, 
2009). 

Yet there seems to be no expectation 
to perfectly match an institution 
with other colleges and universities 
(Anderes, 1 999), hence the reliance 
is on identifying peers or institutions 
with similar characteristics. Institutions 
without existing associations that 
are similar in certain delineating 
factors are deemed as peers (Anderes, 
1999; Trainer, 2008). Regardless, the 
challenge lies with the definition 
of "similar."This is evident from the 


variety of previously reported selection 
methodologies. Some of those 
methodologies — such as nearest 
neighbor and cluster analysis statistical 
techniques, as well as looking at the 
efficacy of only using institutional 
characteristics — are addressed in this 
paper. 

The peer selection described in this 
paper was conducted because the 
existing set of peers was identified 
before the college revised its mission 
and strategic plan. Furthermore, the 
current strategic plan differed from 
previous plans. The new sets of peer 
and aspirant institutions were much 
more aligned with the college's 
mission, goals, and stretch targets 
in the case of the aspirants because 
of several rigorous methodologies 
invoked for the selection. 

This research paper consists of three 
sections. First, the authors give an 
in-depth explanation of the methods 
and overall process of selecting a set 
of institutional peers, peers that are 
relevant and useful for comparative 
analysis and benchmarking. The 
constant exploration, examination, 
and deliberate choice of data and 
information to collect and use are 
evident throughout this paper. The 
peer selection culminates with the 
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two statistical methodologies, nearest 
neighbor to select peer institutions, 
and a two-step cluster analysis 
to determine aspirants. Second, 
alternative methodologies not used 
in the applied research project are 
described. Third, an inventory of 
existing tools is provided that may 
enhance a peer analysis or serve as 
an acceptable substitute. The paper 
concludes with a brief discussion on 
the future direction of peer selection 
and analysis. 

METHOD 

Peer selection is a multi-tier and 
iterative process (McLaughlin et 
al., 2011). This study undertook a 
hybrid approach, amalgamating 
the methodologies of previous peer 
analysis case studies reported in the 
literature. Using multiple methods for 
this applied research project affords a 
practical balance between stakeholder 
judgment and statistics (Trainer, 

2008). The balance achieved better 
credibility than if either was used in 
isolation. Faculty and staff on the peer 
selection design were consulted at 
the beginning of the applied research 
project and regularly at each step. 
Methodology was often adjusted based 
on their insightful suggestions. Hence, 
the mixed method approach used for 
this case study is the result of a failed 
approach at discerning a relevant peer 
group from only a few institutional 
characteristics early in the process and 
the necessity to assert another method. 
Several additional methods were 
used in response to feedback from 
constituents. 

For this peer selection, seven steps 
were undertaken: (1) determination 
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of an initial peer set, (2) collection of 
data on the initial set of institutions, 

(3) variable standardization, (4) parsing 
the initial peer set into several subsets, 
(5) suitability determined by use 
of collected and transformed data 
elements, (6) identification of the best 
variables to use, and (7) selection of 
peers and aspirants. The ground work 
for the aspirant selection was laid by 
the first six steps. The two selection 
methodologies differed only by one 
step. A cluster analysis was substituted 
for the nearest neighbor strategy for 
the selection of aspirants. (Note that 
the institution under investigation will 
be referred to in this paper as the target 
institution.) 

1. Determination of an Initial 
Peer Set 

The initial set of institutions was 
chosen from an original list of private, 
nonprofit institutions that submitted 
data to the Integrated Postsecondary 
Education Data System (IPEDS) from 
the Data Center website (NCES, 201 3b). 
The list was generated using the EZ 
group option (National Center for 
Education Statistics [NCES], 2012). 

Data for these institutions were 
collected for 201 0 and 2011, the most 
recent data available at the time of 
the study. All 4-year private nonprofit 
institutions were included at this 
initial stage if each met the following 
criteria: (a) highest degree awarded 
a bachelor's, a master's, or both, (b) 
enrolled full-time undergraduate 
students, (c) Baccalaureate: Arts & 
Sciences, or Baccalaureate: Balanced 
Arts & Sciences, diverse fields Carnegie 
Classifications, (d) Title IV participant 
(federal financial aid eligibility), 

(e) located in the United States or 


designated as a U.S. Service School 
(e.g., U.S. Naval Academy), and (f) not 
a tribal college. This is on par with 
selection parameters recommended 
by previous studies (Anderes, 1 999). As 
a result of applying these criteria, 285 
institutions were selected. 

2. Collection of Data on the 
Initial Set of Institutions 

Key performance indicators (KPIs) are 
metrics used to measure quality — the 
institution's quality. Quality is defined 
within the context of the institution's 
mission and its priorities. Peer selection 
should be based on information that 
relates to the mission and priorities of 
the institution (Anderes, 1 999; Cohodes 
& Goodman, 201 2). Therefore, the data 
collected for the target institution's KPIs 
would be the information also needed 
for the other institutions. 

Before deciding which KPIs to use, 
some exploratory analysis was 
undertaken. First, each institutional KPI 
was classified based on how it affected 
institutional quality: (a) Influencer- 
Input (e.g., SAT scores, admission 
yield rates), (b) Influencer-Concurrent 
(e.g., academic engagement, crime 
statistics), or (c) Performance Indicator 
- Output (e.g., retention rates, number 
of conferred bachelor's degrees). Next, 
a group of faculty and staff were asked 
to rate the importance of each KPI as 
it related to institutional quality. Data 
slated to be collected, hence, were 
informed by the KPI classifications 
and the faculty and staff importance 
ratings. 

The caveat was that these data had to 
be readily available and easily accessed 
for the other 285 institutions. For this 


institution, KPI data were gathered 
from a variety of sources, primarily 
from national consortiums, surveys, 
and IPEDS. Data from the former 
included (a) National Survey of Student 
Engagement (NSSE) benchmarks, (b) 
American Association of University 
Professors (AAUP) faculty salary data, 

(c) Noel Levitz Student Satisfaction 
Inventory (NLSSI), and (d) U.S. News 
& World Report rankings (U.S. News 
& World Report, 201 1). However, 
not all 285 institutions participate in 
the NSSE or NLSSI, and AAUP data 
at the individual institution level 
are not available. Consequently, the 
variables used in the peer analysis 
were primarily sourced from IPEDS 
and the U.S. News & World Report 
rankings. Those variables are shown in 
Appendix A. Descriptions of each are 
provided in Appendix B. Examples of 
data that were collected as a result of 
availability include tuition, total price of 
attendance, total enrollment, financial 
expenditures, SAT scores, admit yield, 
and quality of faculty. 

3. Variable Standardization 

After a preliminary examination of 
the data, it was discovered that the 
enrollment and number of applicants 
at the target institution were almost 
double that of most of the other 
institutions. To control for institutional 
size, each institution's reported full- 
time equivalent (FTE) enrollment was 
divided into some of the data elements 
to eliminate the bias that may result 
from differences in enrollment size 
(Gater, 2003; Huxley, 2009). Examples of 
data elements that were standardized 
by dividing by FTE include the number 
of conferred bachelor's degrees, 
number of applicants, unduplicated 


annual enrollment, instructional 
expenses, and endowment. 

The researchers had access to both 
full-time and part-time faculty counts. 
These were combined into one data 
element — the proportion of full-time 
faculty to full-time plus part-time 
faculty. Some variables were not 
converted. Retention and graduation 
rates were not altered. The percent 
of classes with 20 or fewer students 
was not changed; the data element 
is not affected by the differences in 
size of enrollment among institutions. 
Likewise, admissions yield and alumni 
giving rates, expressed as proportions, 
were not transformed. Faculty salaries 
were already reported as an average 
and, therefore, were not changed. 
Similarly, downloaded SAT score 
percentiles remained unaltered. The 
percent of transfer students was 
classified into two categories — low and 
high. 

4. Parsing the Initial Peer Set 
into Several Subsets 

A workable peer group size was sought 
to abet further analysis and peer 
selection. Depending on purpose, a 
reasonable peer group size has been 
identified to be between 5 and 40 
institutions (McLaughlin et al., 2011). 

Five subgroups were assembled based 
on institutional characteristics gleaned 
from the school's Carnegie Classification: 
(a) Catholic affiliation, (b) primarily 
baccalaureate, (c) highly residential, (d) 
low proportion of transfer students, and 
(e) more selective (Carnegie Foundation, 
201 0). Previously identified peer and 
competitor groups formed the basis for 
these subgroup categories. Aggregate 
information was compiled for the 


data elements collected in Step 2, and 
standardized if appropriate, for all 285 
institutions and the five institution 
subsets. 

5. Suitability Determined 
by Use of Collected and 
Transformed Data Elements 

To better clarify the many comparisons 
to be made in the following steps, 
a mean was computed for the data 
elements listed in Appendix A and 
standardized when applicable for 
each subgroup. These means were 
compared to the target institution's 
data. 

The target institution was similar 
to the subgroups in some aspects 
but noticeably different on other 
parameters. As such, no group 
clearly emerged as comparable. For 
example, total price of attendance 
was similar to the target institution 
for all subgroups, but the target 
institution had better 1-year retention 
rates and 6-year graduation rates for 
all subgroups except for the more- 
selective subgroup. On the other hand, 
the target institution had a smaller 
proportion of full-time faculty and 
alumni giving rate than the institutions 
in the more selective sub-group. 
Although no tests of significance 
were used, statistical testing could 
have quantified these differences 
and possibly better determined the 
adequacy of each subgroup as a 
potential peer set. 

Therefore, an additional reference 
group was formed by combining three 
of the above criteria: (a) low proportion 
of transfer students, (b) highly 
residential, and (c) more selective. 
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Again, no definitive peer set emerged. 

As these comparisons demonstrate, 
selecting an initial set of peers based on 
institutional characteristics may seem to 
be a practical and logical approach, but 
can be ineffectual (Shin, 2009). 

6. Identification of the Best 
Variables to Use 

The committee of faculty and staff 
was instructed to identify the KPI 
performance measures that best 
aligned with the priorities of the 
college. Three KPIs were identified: 

(a) 1-year retention rate, (b) 6-year 
graduation rate, and (c) proportion 
of students to bachelor's degrees 
awarded. Variables for peer selection 
would be determined by their 
predictive power of the three KPI 
performance measures. 

Several regression models were 
identified for each KPI. This was 
accomplished in two phases. First, 
the data elements were classified into 
five categories: (a) admissions, (b) 
faculty, (c) enrollment, (d) institutional 
characteristics, and (e) finance. Ordinary 
least square (OLS) regression models 
using a single-step enter method in 
SPSS were compiled separately for the 
five variable categories for each KPI, 
a total of fifteen models. Because the 
analysis was still exploratory at this 
stage, the single-step enter method 
was preferred over other models. This 
afforded the inclusion of all category 
variables in the model, enabling 
comparisons among the variables (SPSS, 
2008). Directed by previous research, the 
resulting regression coefficients were 
the determinants of data elements that 
would be used for peer selection (Horn, 
2008). 


In the second phase, an overall 
regression model for each KPI was 
computed using the best predictor 
or predictors from each of the five 
category regressions. The variable 
with the smallest significance level 
associated with the standardized beta 
coefficient was deemed to be the 
best predictor. The significance of a 
beta weight indicates if the variable 
is a predictor relative to the variable's 


absence in the model (Cohen & Cohen, 
1 983). In most cases, only one variable 
from each category was chosen for 
the three overall models because of 
the relatively high correlations among 
the variables within their categories. 

In effect, this reduced the relatedness 
or redundancy of the variables in the 
three overall models. It also maximized 
the potential predictive strength of 
each variable. Additionally, a balance 


Table 1 . Overall OLS Regression Models for the Three Performance Indicators: 
Ratio of Conferred Bachelor's Degrees to FTE, 1-Year Retention Rate, and 6-Year 
Graduation Rate 


Category 

Variable* 

Standardized 
Beta Weight 

Ratio of Conferred Bachelor's Degrees to FTE 


Admissions 

25th Percentile Mathematics SAT 

.348* 

Faculty 

Average Faculty Salary 

-.142 

Enrollment 

Estimated Fall Enrollment to FTE 

-.053 

Institutional Characteristics 

Selectivity 

.282** 

Finance 

Instructional Expenses 

.166 

1-Year Retention Rate 

Admissions 

25th Percentile Mathematics SAT 

.465*** 

Faculty 

Average Faculty Salary 

.135 

Enrollment 

FTE 

.064 

Institutional Characteristics 

Selectivity 

.301*** 

Finance 

Instructional Expenses 

.065 

6-Year Graduation Rate 

Admissions 

Percent of First Time Federal 

Grant Aid Students 

-.145** 

Faculty 

Average Faculty Salary 

.211** 

Enrollment 

FTE 

.090 

Institutional Characteristics 

Selectivity 

.178*** 


Proportion of Transfer Students 

-.104** 

Finance 

Total Price of Attendance 

.007 


Instructional Expenses 

.224*** 


Alumni Giving Rate 

.186* 

* p < .05, **p<.01,***p<.001. 
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of institutional metrics for peer 
selection was sought by using the 
best predictors from each of the five 
variable categories rather than five best 
predictors regardless of category. The 
best predictors for each KPI regression 
model by category are listed in Table 1 . 

Two different admissions data elements 
were identified for the overall models. 
The 25th percentile Mathematics SAT 
variable was the best predictor for 
both the ratio of conferred bachelor's 
degree to FTE and the 1 -year retention 
rate overall models. For the 6-year 
graduation rate, percent of first-time 
federal grant aid students was best. 

Curiously, average faculty salary 
reigned supreme for all three overall 
models. In fact, this faculty data 
element was the only significant data 
element for the bachelor's degree 
to FTE regression model ((3 = .400, 
p < .001). The standardized beta 
weight far exceeded the other faculty 
data elements in the 1-year retention 
rate model ((3 = .622, p < .001). The 
faculty data element with the next- 
largest standardized beta weight 
in the 1-year retention rate model, 
percent of full-time instructors, was 
perceptibly smaller ((3 = .1 65, p < .01 ). 
Similar results were observed for the 
6-year graduation rate model with the 
standardized beta weight for percent 
of full-time instructors smaller than 
the average faculty salary standardized 
beta weight ((3 = .1 76, p < .001 , (3 = .630, 
p < .001, respectively). 

In the enrollment category, FTE was the 
best predictor for two of the models: 
1-year retention rate and 6-year 
graduation rates. The transformed 


variable, estimated fall enrollment 
to FTE, had the best beta coefficient 
significance level for the ratio of 
conferred bachelor's degree to FTE. 

Not surprisingly, selectivity was the 
institutional characteristic with the best 
beta coefficient significance level for all 
three overall models. The proportion of 
transfer students was also an equally 
significant institutional characteristic 
for the 6-year graduation rate overall 
model. For the finance category, 
instructional expenses bubbled to the 
top for ratio of conferred bachelor's 
degree to FTE and 1-year retention 
rate models. Three finance variables 
fared best for the 6-year graduation 
model: (a) total price of attendance, (b) 
instructional expenses, and (c) alumni 
giving rate. 

The data elements, FTE, and estimated 
fall enrollment to FTE, are highly 
correlated, (r = .874, p < .001 ). The 
latter data element may be perceived 
as confusing and is not as commonly 
used as FTE. Therefore, estimated fall 
enrollment to FTE was eliminated 
from further consideration. Nine data 
elements, the strongest predictors 
of the three KPIs, remained and were 
the basis for the analysis in the next 
and final step: (a) 25th percentile 
Mathematics SAT, (b) percent of first- 
time students receiving federal grant 
aid, (c) average faculty salary, (d) FTE, 

(e) selectivity, (f) proportion of transfer 
students, (g) instructional expenses, 

(h) total price of attendance, and 

(i) alumni giving rate. In short, nine 
variables that were statistically the best 
predictors of the college's priorities 

as deemed by a consensus of faculty 
and staff will be the basis of the peer 
selection. Moreover, these predictors 


are representative of the inputs and 
outputs that affect institutional quality. 

7. Selection of Peers and 
Aspirants 

Use of the nearest neighbor statistical 
technique to compute proximity index. 
Nearest neighbor methodology is a 
multi-step process: (a) determining the 
most relevant parameter calculations, 
(b) computing the numerical 
difference between the reference and 
target institutions on each of those 
parameters, and (c) deciding the range 
that constitutes a proximate or"near 
neighbor" difference. As such, peer 
institutions are determined by having 
metrics that are proximate to the target 
institution (McLaughlin et al., 2011). 

For the peer selection, the numeric 
difference between the target and 
each comparison institution was 
determined for the nine variables. In 
turn, these differences determined 
peer proximity or nearest neighbor. A 
proximity score was compiled using the 
standard deviation of each predictor to 
measure nearness as shown in Figure 1 . 
Specifically, a proximity score of 1 was 
assigned to any institution that was 
between one-half and one standard 
deviation of target institution's metric, 
and a score of 2 if the institution was 
within one-half standard deviation. The 
average of the nine equally weighted 
proximity scores derived the proximity 
index. Generically, these computations 
can be represented by two simple 
equations: 
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Figure 1 . Proximity and Aspirant Index Numeric Assignments for Differences 
Between Reference College and Institution 



-2a -la -Vio 0 +34a +1 a +2a 


Figure 2. Proximity Index and Carnegie Classification for Near Peer and Almost 
Near Peer Institutions 


Proximity Index for 

Near Peers and Almost Near Peers Institutions 



Proxmity Index 


PSvarl — (TIvarx — CIvarx)/ SDvarx (1) 

varx e{l, ... 9} 

Plinstitution = average (PSvarl . . . PSvar9) (2) 

institution e{l, . . . 285} 

Where: 

PS = Proximity Score 
PI = Proximity Index 
TI = Target Institution 
Cl = Comparison Institution 
Varl - Var9 = Predictors 

0 reassigned to PS when: PS > 1 or PS < -1 

1 reassigned to PS when: -1 < PS < -.5 or .5 < PS < 1 

2 reassigned to PS when: — .5 < PS < .5 

This case study departs from the nearest 
neighbor methodology (McLaughlin 
et al., 201 1 ). A small number of data 
elements was used to compute the 
proximity index, which is an aggregate 
score of the nine predictor variables. 
Furthermore, the variables were 
weighted equally, a decision made by 
the researchers. Nevertheless, these 
changes are warranted. A large number 
of variables that are highly correlated 
may make the meaning of the proximity 
index difficult to decipher and obscure 
the advantage of its use (Gater, 2003; 
Lorr, 1983). 

The average proximity scores — the 
proximity indices — for the 285 
institutions range from 0 to 1 .78. 
Examining first the range of proximity 
indices and then the resulting 
percentiles for these institutions, two 
peer sets emerged. The first set of 
peers, given the moniker "near peers," 
comprised the 19 institutions having 
proximity indices corresponding to the 
95th percentile or higher. Another 1 9 
institutions constituted the next tier of 


peer institutions, dubbed "almost near 
peers."These almost near peers had 
proximity indices between the 90th 
and 95th percentiles. These two sets of 
peers are shown in Appendix C. 

As seen in Figure 2, the basic Carnegie 
Classification for four of the near 
peer institutions and three of the 
almost near peers are Baccalaureate: 


Diverse Fields. The remaining peers are 
Baccalaureate: Arts and Sciences, which 
is the same as the target institution. 

Aspirant selection determined by cluster 
analysis. Most studies evaluated for 
this paper had a singular focus (peer 
selection or aspirant identification, 
but not both). Accordingly, none 
distinguished the differences between 
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peer group formation methodology 
and the process to determine aspirant 
institutions. In this respect, this case 
study differs from previous research. 

A different statistical method — cluster 
analysis — was used to determine a list 
of aspirant institutions. That said, the 
same nine predictor variables and KPIs 
used for the nearest neighbor analysis 
were used for the aspirant analysis. 

In preparation, quartile cut scores were 
identified for each KPI for the initial 
set of institutions. Institutions were 
then assigned to their corresponding 
quartiles, one for each KPI. Next, a 
two-step cluster analysis using the 
likelihood distance method was 
performed for each KPI. This was 
accomplished by using the best 
predictors for each KPI from the 
five categories listed in Table 1 . In a 
two-step cluster analysis, individual 
institutions are consecutively 


combined to form clusters subsequent 
to an initial pass (SPSS, 2008). Figure 3 
shows the three cluster panels, one for 
each KPI. Each panel consists of two 
or three columns, one for each cluster. 
Listed in each column are the predictor 
means or variables used to construct 
the cluster followed by the average 
KPI quartile category assignment, 
designated as the evaluation field. 

The ratio of conferred bachelor's 
degree to FTE and 1-year retention rate 
KPIs yielded three clusters, whereas the 
6-year graduation KPI cluster analysis 
was less discerning and produced only 
two clusters. For these first two KPIs, 
approximately one-third (35.4%) of the 
institutions were in the cluster with the 
best KPI quartile category average, the 
aspirant cluster. The 6-year graduation 
rate cluster rate was less distinguishing. 
For this KPI, the aspirant cluster 
represented more than half (52.5%) of 


the institutions as possible aspirants. 

The same cluster variables were used 
for the ratio of conferred bachelor's 
degrees to FTE and 1-year retention 
rate KPI cluster analyses. The cluster 
assignments among the institutions 
were the same for these two cluster 
analyses and, therefore, the cluster 
analyses are essentially identical. 
Because the underlying goal is to 
identify a reasonable number of 
aspirants, and two of the cluster 
analyses produced a smaller identical 
set of aspirant institutions than 
the third, the clusters from the two 
identical clusters were examined 
further. 

Figure 4 depicts the distance of 
the predictor variables among the 
clusters graphically. Since the clusters 
were redundant, only one set of 
clusters is portrayed but with seven 


Figure 3. Cluster Size and Means for Each Cluster Associated with the Evaluation Fields 

Note: Orange highlight indicates best or "aspirant" cluster. 
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KPI: 1 Year Retention Rate | 
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3. ABOVE the Mean: 
Bachelor's Degree to FTE 

2. AT the Mean: 
Bachelor's Degree to FTE 

1. BELOW the Mean: 
Bachelor's Degree to FTE 


Description 

3. ABOVE the Mean: 1 

Year Retention Rate 

2. AT the Mean: 1 Year 

Retention Rate 

1. BELOW the Mean: 1 

Year Retention Rate 

Size 

35.4% (n=80) 

44.47% (n=101) 

19.9% (n=45) 


Size 

35.4% (n=80) 

44.47% (n=101) 

19.9% (n=45) 

Inputs 

Selectivity: More 
Selective 

Selectivity: Selective 

Selectivity: Inclusive 


Inputs 

Selectivity: More 
Selective 

Selectivity: Selective 

Selectivity: Inclusive 

SAT Math 25th Percentile 
Score, 2010-11: Mean= 
571.55 

SAT Math 25th Percentile 
Score, 2010-11: Mean= 
465.15 

SAT Math 25th Percentile 

Score, 2010-11: Mean= 
406.44 

SAT Math 25th Percentile 

Score, 2010-11: Mean= 
571.55 

SAT Math 25th Percentile 

Score, 2010-11: Mean= 
465.15 

SAT Math 25th Percentile 
Score, 2010-11: Mean= 

406.44 

Instructional Expenses 
per FTE: Mean= 
$15,724.06 

Instructional Expenses 
per FTE, Mean-= 
$7,628.18 

Instructional Expenses 
per FTE: Mean= 
$5,782.85 

Instructional Expenses 
per FTE: Mean= 
$15,724.06 

Instructional Expenses 
per FTE, Mean= 
$7,628.18 

Instructional Expenses 
per FTE: Mean= 
$5,782.85 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$76,156.18 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$56,271.03 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$51,876.53 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$76,156.18 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$56,271.03 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$51,876.53 

FTE, Fall 2010: Mean= 
1,966.21 

FTE, Fall 2010: Mean= 
1,584.70 

FTE, Fall 2010: Mean= 
1,455.20 

FTE, Fall 2010: Mean= 
1,966.21 

FTE, Fall 2010: Mean= 
1,584.70 

FTE, Fall 2010: Mean= 
1,455.20 

Evaluation 
Field (KPI) 

Bachelor's Degree to FTE 

Bachelor's Degree to FTE 

Bachelor's Degree to FTE 


Evaluation 
Field (KPI) 

1 Year Retention Rate 

1 Year Retention Rate 

1 Year Retention Rate 


Bold indicates most frequent category or mean for cluster. Darker shades indicate more Bold indicates most frequent category or mean for cluster. Darker shades indicate more 

important inputs important inputs 


| KPI: 6 Year Graduation Rate 

Cluster 

2 

1 

Description 

2. ABOVE the Mean: 6 

Year Graduation Rate 

1. BELOW the Mean: 6 

Year Graduation Rate 

Size 

47.5% (n=96) 

52.5% (n=106) ' 


Selectivity: More 
Selective 

Selectivity: Selective 


Total Price (In-District 
Students), 2011-12: 
Mean= $49,411.96 

Total Price (In-District 
Students), 2011-12: 
Mean= $37,474.40 


Instructional Expenses per 
FTE: Mean= $15,663.95 

Instructional Expenses per 
FTE, Mean-= $7,749.1 1 

Inputs 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$76,441 .92 

Average Full-Time Faculty 
Salary (All ranks): Mean= 
$56,838.32 ) 


% FT Students Receiving 
Federal Aid: Mean= 

22.02% 

% FT Students Receiving 
Federal Aid: Mean= 

41.75% 


Alumni Giving Rate: 
Mean= 31.11 

Alumni Giving Rate: 
Mean= 18.31 


Transfer Proportion: Low 
Transfer 

Transfer Proportion: Low 
Transfer 


FTE, Fall 2010: Mean= 
1,964.60 

FTE, Fall 2010: Mean= 
1,611.24 

Evaluation 
Field (KPI) 

6 Year Graduation Rate 

6 Year Graduation Rate 


Bold indicates most frequent category or mean for cluster. Darker 
shades indicate more important inputs 
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cluster variables from the two cluster 
models. Specifically, the seven cluster 
continuous scaled variables were 
each separately converted to quartile 
categories similar to the transformation 
made for the three KPIs (McLaughlin 
et al., 2001; Merisotis & Shedd, 2001). 
Selectivity and proportion of transfer 
students, both ordinal scale variables, 
were not included in this graphical 
representation. The resulting average 
quartile category for each cluster for the 
seven cluster variables and three KPIs 
is plotted using the cluster categories 
assigned to the institution. The 
distinction among clusters is noticeable 
and much clearer in the radar chart in 
Figure 4 than discerned in the separate 
cluster panels in Figure 2. 

The aspirant cluster has the largest 
quartile averages for every predictor 
except for percent of new students 
receiving federal grant aid. Conversely, 
Cluster 1 has the smallest average for 
nine of the ten plotted variables. This 
cluster has the largest quartile average 
for the percent of new students 
receiving federal grant aid. 

Although the two identical cluster 
analyses yielded a smaller set of 
aspirant institutions than the 6-year 
graduation rate cluster analyses, not all 
were part of the latter aspirant group. 
Therefore, only the 52 institutions that 
were in all three aspirant clusters would 
be considered as potential aspirants. 
Somewhat unmanageable in size, an 
aspirant index was computed for these 
52 schools. In concept, the aspirant 
index is similar to the proximity index 
with four germane distinctions: 

First, nine KPI predictors were the 
basis of the proximity index, but the 


Figure 4. Quartile Means for Each Cluster 


proximity scores for the actual KPIs 
were excluded from the proximity 
index. However, the aspirant scores 
for each KPI were included in the 
aspirant index calculations. Second, the 
proximity index gave more credence 
to small differences, and the aspirant 
index more weight to large positive 
differences. For the proximity index, 
larger numerical values were assigned 
to institutions that were close to 
the target institution than to those 
that were not. However, the aspirant 
schema awarded larger absolute 
values to large differences between 
the target and comparison institutions 
than those that had small differences. 
Third, the direction of that difference 
is unimportant in the proximity index 
calculation but is at the heart of the 
aspirant index calculation. That is, if 
the aspirant metric was greater than 
the target institution's value, a positive 
aspirant score was assigned. Fourth, the 
standard deviations used to determine 
aspirant scores were only compiled for 
the 52 schools in the aspirant clusters. 


As such, standard deviation was 
computed for each KPI for the aspirant 
cluster institutions. An aspirant score 
of one was assigned to any institution 
that was between one-half and one 
standard deviation above the target 
institution's metric, and a score of 
two was given if the institution was 
greater than one standard deviation. 
Correspondingly, if the institution's 
value was between one-half and one 
standard deviation below the target 
institution's score, a negative one was 
assigned, and if the institution was 
greater than one standard deviation 
below the target institution a negative 
two was assigned. A zero was given to 
an institution's metric within one-half 
the standard deviation above or below 
the target institution. Figure 1 provides 
a visual depiction of the numerical 
assignments. 

The average of the three equally 
weighted aspirant scores comprised 
the aspirant index. The equations used 
in the aspirant score described above 
and aspirant index computations 
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are similar to the proximity index 
calculations. That is, 

ASkpix = (TIkpix - CIkpix) / SDkpix ^ 

X £{1,2,3} 

Alaspirant institution = average (ASkPII . . . ASKPI3) 

aspirant institution e{l, . . . 52} (4) 

Where: 

AS = Aspirant Score 
AI = Aspirant Index 
TI = Target Institution 
Cl = Comparison Institution 
Varl - Var9 = Predictors 

-2 reassigned to AS when AS: < -1 
-1 reassigned to AS when AS: -1 < AS < -.5 

0 reassigned to AS when AS: -.5 < AS < .5 

1 reassigned to AS when AS: .5 < AS < 1 

2 reassigned to AS when AS: > 1 

Results were as follows: 1 2 institutions 
had a negative aspirant index and were 
removed from the initial aspirant list, 27 
institutions posted a zero aspirational 
index, and 1 3 institutions of the 
preliminary 52 aspirant institutions 
had an aspirant index greater than 
zero. The latter constituted the aspirant 
list or Tier I aspirant institutions. The 
former set of institutions is ancillary 
and comprises theTier II aspirant list. 
These lists are shown in Appendix C. 
The basic Carnegie Classification for all 
institutions on the two aspirant lists is 
Baccalaureate: Arts and Sciences. 

OTHER REFERENCE 
GROUP SELECTION 
METHODOLOGIES 

Deciding on the most appropriate 
method requires both the knowledge 
of the statistical procedure and the 


purpose of the eventual comparison(s). 
Two statistical methodologies were 
used for this case study: (a) nearest 
neighbor and (b) two-step cluster 
analysis. However, other techniques 
should be considered either singularly, 
or as a mixed methods approach as 
with this case study. Other techniques 
may be better suited to the anticipated 
purposes of the peer analysis. Although 
not exhaustive, other techniques 
documented in the literature are 
provided below. This listing provides 
a more comprehensive collection of 
peer and aspirant selection techniques 
than has been discussed thus far in this 
paper and in the literature. 

Cluster analysis. At least two cluster 
analysis techniques have been 
employed to determine peer 
institutions. First, as the name implies, 
two-step cluster analysis entails two 
iterations — one to decipher the 
cluster and corresponding cluster 
centers and another to determine 
cluster assignment among institutions. 
Second, hierarchical cluster analysis can 
be deployed when a small number of 
initial institutions are being considered. 
In this method, the distance between 
institutions on a set of parameters 
is computed (Horn, 2008). Euclidean 
distance and correlations are the 
most common, although the latter 
is discouraged (Lorr, 1 983). Based 
on these distance designations, the 
researcher can determine clusters by 
assigning membership, often manually. 

Data envelopment analysis (DEA). 

This statistical procedure determines 
the most efficient institutions, often 
indicated by financial indicators 
(Eckles, 2009). A disadvantage of this 
selection tool is that the identification 
of benchmark institutions is relative to 
the original list of selected institutions 


regardless of their actual efficiency 
(Taylor & Harris, 2004). 

Discriminant analysis. This statistical 
technique classifies institutions into 
one or more mutually exclusive 
groups. Accomplished in two steps, 
a classification rule is first developed 
using institutions for which group 
membership is known. Next, 
institutions are sorted into groups 
based on the classification rule (SPSS, 
2008). The first phase may render 
this technique unworkable. For peer 
selection, group membership is seldom 
established or known, rendering this 
technique impractical (Huxley, 2009). 

Factor analysis. Institutions are 
classified by factors determined by 
the correlations or covariances among 
institutional parameters. As with 
discriminant analysis, prior knowledge 
of the institutions and the associations 
among institutional parameters is 
necessary. 

Nearest neighbor. The determination of 
the best matches or nearest neighbors 
varies but the crux of this methodology 
is to decipher the extent to which an 
institution is a peer (McLaughlin et al., 

201 l).This is accomplished by computing 
the distance between institutions on 
targeted predetermined parameters. 

Subject matter experts (SME). Engaging 
faculty and staff that have a vested 
interest in assembling a set of peers 
not only is a sound method to validate 
a proposed set of peer institutions, 
but also may increase the likelihood 
of the reference group's acceptability 
and use. This method is recommended 
for specific purposes rather than as a 
general institutional peer selection. 
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Variable match. Reference group 
formation does not necessarily require 
sophisticated statistical procedures, 
making this technique popular. In 
fact, alignment by a few institutional 
parameters may be sufficient (Anderes, 

1 999). This review should start with 
the mission of each institution under 
consideration. Subsequently, an 
examination of comparability of other 
institutional characteristics should 
ensue. This may include enrollment 
size, degree level and program mix, 
type of students served, setting (urban, 
suburban, rural), expenditures, and 
selectivity. 

EXTERNAL PEER 
SELECTION TOOLS 

Web peer selection applications have 
proliferated recently. The authors posit 
that these tools can also be useful and 
certainly provide data and comparative 
analysis beyond that described for this 
applied research project. In fact, many 
of the applications were discovered 
during the design of this case study 
and some were used for preliminary 
investigations. 

The purpose of including a brief 
mention of these web sources is 
threefold. First, knowing the existence 
of these resources may save time 
and the effort of conducting a peer 
selection from scratch. Most of these 
websites have comparative capabilities, 
and therefore may be perfectly capable 
and sufficient for the intended purpose. 
Second, awareness of these resources 
equates to knowing where to locate 
needed data and information. In turn, 
this will help to lessen the time needed 
for one of the most time-consuming 


of the steps of a peer selection: finding 
the data elements and acquiring that 
information for the institutions under 
consideration. Third, exploration of the 
resources could uncover additional 
data elements not mentioned in this 
applied research study. For ease of 
reference, brief descriptions and the 
web addresses for each are provided in 
Appendix D. 

CONCLUSIONS 

Self-labeled as mixed methods, eight 
steps in total were needed to select a 
set of institutional peers and aspirants. 
This was partially due to modifications 
made during the study based on 
stakeholder feedback, and in part 
due to trial and error. For example, 
the subsets of peers in Step 4 were 
collectively subpar, and were not 
comparable to the target institution. 
Subsequently, however, a superior 
set of peers was determined by a 
multi-layered statistical approach that 
helps to unearth the institutional 
characteristics that best aligned with 
the college's priorities. To that end, 
the following techniques were used: 

(a) data element standardization, (b) 
parsing standardized data elements 
into several categories, (c) using several 
regression models to determine the 
standardized data elements that are 
best correlated with key institutional 
attributes, (d) computing proximity 
scores with the standardized data 
elements determined from the 
regression models to be the most 
appropriate, and (e) compiling a 
cumulative proximity index. This study 
does not, per se, add to the list of 
selection methodologies, but rather 
reinforces the value of using multiple 


methods. Furthermore, this study 
demonstrates that a multi-method 
approach is preferable to a single- 
method approach. 

The change in process during the 
course of the study illustrates both the 
ease and the flexibility of the process 
itself. Importantly, examining the set of 
institutions gleaned by each method 
affords both a comparison of the 
appropriateness of each institution as 
a peer and the set of institutions as a 
reasonable peer group. The researchers 
conjecture that the latter analyses 
further strengthen the utility of the 
final set of peers and/or aspirants. 

Therefore, and as this study 
demonstrates, peer selection based 
on institutional characteristics alone 
is inferior to a multi-staged approach. 
Determining institutional peers based 
on both the parameters that reflect 
institutional performance priorities 
and data elements that are indicative 
of those priorities may be a better 
approach. The engagement of faculty 
and staff to identify the information 
and procedures used in this applied 
research project helped to select peers 
that were better aligned with those 
institutional imperatives. Moreover, the 
inclusiveness of the process improved 
the credibility and eventual use of the 
final peer and aspirant lists. 
Tangentially, the inclusion of several 
stakeholders in the process had the 
added benefit of debunking the 
perceived superiority of several data 
elements. For example, previous to 
this study endowment, percentage 
of faculty holding terminal degrees, 
percentage of classes enrolling fewer 
than 20 students, and total price 
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of students living on campus were 
frequently cited as determinants of 
student success. Furthermore, the U.S. 
News & World Report rankings are 
often cited as an indicator of quality. 

As this study has shown for this set 
of initial peers, these institutional 
characteristics all contribute to an 
institution's U.S. News & World Report 
ranking but are not necessarily 
indicative of quality. 

Admittedly, the data elements, 
especially the variables chosen for the 
peer and aspirant selection statistical 
procedures, were highly correlated. 

For example, the correlation between 
instructional expenses per FTE and 
alumni giving rate was fairly high 
(r = .76, p < .001 ) as was the correlation 
between average faculty salaries 
and percent of full-time faculty with 
terminal degree (r = .617, p < .001). 

The five variable classifications and the 
designation of one or a few variables 
from each in the final regression 
models were designed to mitigate this 
phenomenon. Hence, the potential 
multi-collinearity among the variables 
was diminished somewhat by 
instituting five categories and limiting 
variable inclusion. 

Availability of data from IPEDS and 
other sources continues to expand. 

In addition, linked information from 
diverse resources is readily available 
(Trainer, 2008). This expansion of 
data access may be the provocation 
responsible for the increase in 
the popularity of peer analysis. 
Additionally, institutions and state 
systems of higher education have 
responded to public scrutiny by 
using peer comparisons, a familiar 
embodiment of quality. 


RECOMMENDATIONS 

The importance of the selection of data 
elements and statistical techniques 
cannot be overstated. It is imperative 
to consider both the type of institution 
and the purpose of the peer selection 
(Shin, 2009). In short, five factors that 
should be considered before selecting 
a set of peers were identified in this 
applied research project: (a) target 
institution mission and institutional 
type, (b) ultimate purpose of peer 
analysis, (c) ease of the collection of 
data and information, (d) stakeholder 
understanding and perception of 
alignment to campus priorities, and 
(e) timing. To the last point, use of 
historical information, data trends, or 
the most current data are posited as 
options but may not all be appropriate. 

Engaging multiple methods may 
address the limitations of a single 
approach. As with this study, a mixed- 
methods paradigm yielded the most 
appropriate fit of potentially disparate 
purposes of the peer and aspirant 
lists. Moreover, the iterative process 
revealed the weakness of selecting 
peer institutions on appearances or 
similar characteristics versus choosing 
peers based on performance and 
indicators of quality. Collectively, each 
peer and the peer set as a whole can be 
explicitly justified. Furthermore, the list 
is validated by informed and interested 
stakeholders. Likewise, the choice of 
aspirant institutions is unambiguous, 
based on institutional providence. 
Importantly, mixed-methodologies 
glean peers and aspirants that are 
meaningful and practical. 

Despite data availability, data 
element selection, and breadth of 


methodologies, the cluster analysis 
identified more than 50 aspirant 
institutions, an impractical size. 

This reinforces that even the most 
sophisticated statistical techniques 
and unfettered availability of data can 
replace neither a clearly stated purpose 
of the comparison nor input from 
various stakeholder groups. The former 
provides irreplaceable selection criteria, 
and the latter helps to confirm the 
legitimacy of institutions as members 
of the peer or aspirant groups. For this 
applied research study, an aspirant 
index was devised to further pinpoint 
a reasonable number of aspirants. 
Following the logic of the nearest 
neighbor, the aspirant index was 
changed slightly to identify the best- 
performing institutions. Because of its 
similarity to the nearest neighbor and 
its simplicity, the aspirant index should 
be considered as another potential 
valuable statistical technique. 

Published studies about peer selection 
are scarce; as a result, clear direction on 
peer selection methodology is limited. 
Peer selection models, differentiated 
by institutional type and function, 
could evolve with additional evidence. 
Furthermore, the impact of peer 
comparisons on institutional quality 
and improvement is unknown. Little 
guidance exists on the evaluation of 
peer selection and subsequent peer 
comparisons (Powell, Gilleland Suitt, 

& Pearson, 201 2). Further research 
should evaluate the effects of peer 
comparisons, if any, on institutional 
quality. Only with further investigation 
will the impact of peer comparisons on 
institutional quality be known. 
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Appendix B. Data Element Used 
for Peer and Aspirant Selection: 
Descriptions 

Admissions 

Admit Yield: Number of enrolled divided 
by number admitted. 

Number of Applicants, Total: Number of 
first-time, degree- or certificate-seeking 
undergraduate students who applied 
(full or part time). Includes early 
decision, early action, and students 
who began studies during the summer 
prior to that fall. 

Percent of Applicants Admitted: Number 
of admitted divided by total applicants. 
SAT Critical Reading 25th Percentile 
Score: Includes new students admitted 
the summer prior to that fall. 

SAT Critical Reading 75th Percentile 
Score: Includes new students admitted 
the summer prior to that fall. 

SAT Math 25th Percentile Score: Includes 
new students admitted the summer 
prior to that fall. 
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Appendix A. Data Elements Used for Peer and Aspirant Selection: 
Time Frame, Indicator Type, and Source 


Variable 

Time Frame 

Indicator Type 

Indicator Source 

Admit Yield 

2011-12 

Admissions 

IPEDS 

Number of Applicants, Total 

2011-12 

Admissions 

IPEDS 

Percent of Applicants Admitted 

2011-12 

Admissions 

IPEDS 

SAT Critical Reading 25th Percentile Score 

2010-11 

Admissions 

IPEDS 

SAT Critical Reading 75th Percentile Score 

2010-11 

Admissions 

IPEDS 

SAT Math 25th Percentile Score 

2010-11 

Admissions 

IPEDS 

SAT Math 75th Percentile Score 

2010-11 

Admissions 

IPEDS 

Bachelors Degrees Conferred 

2010-11 

Completions 

IPEDS 

Estimated Fall Enrollment 

Fall 2010 

Enrollment 

IPEDS 

Full-Time Equivalent (FTE) 

Fall 2010 

Enrollment 

IPEDS 

Total Enrollment, Unduplicated 

2010-11 

Enrollment 

IPEDS 

Percentage of Classes Enrolling Fewer Than 20 
Students 

2011-12 

Enrollment 

US News & World 
Report 

Average Salary Equated to 9-Month Contracts of 
Full-Time Instructional Staff- All Ranks 

2011-12 

Faculty 

IPEDS 

Full-Time Primary Instruction Head Count 

Fall 2011 

Faculty 

IPEDS 

Part-Time Primary Instruction Head Count 

Fall 2011 

Faculty 

IPEDS 

Percentage of Faculty Holding Terminal Degrees 

2011-12 

Faculty 

US News & World 
Report 

Endowment (FASB) 

2009-10 

Financial 

IPEDS 

Instructional Expenses Per FTE (FASB) 

2009-10 

Financial 

IPEDS 

Tuition- Total Price for In-District Students Living 
on Campus 

2011-12 

Financial 

IPEDS 

Alumni Giving Rate 

2011-12 

Financial 

US News & World 
Report 

Percent of Full-Time Undergraduates Receiving 
Federal Grant Aid 

2010-11 

Financial Aid 

IPEDS 

Carnegie Classification- Basic (Arts & Sciences 
or Diverse Fields) 

— 

Institutional Char- 
acteristic 

IPEDS 

Carnegie Classification- Enrollment Size & 

Setting 

— 

Institutional Char- 
acteristic 

IPEDS 

Carnegie Classification- Undergraduate Profile 
(Transfer and Full-Time proportions) 

— 

Institutional Char- 
acteristic 

IPEDS 

Geographic Region 

— 

Institutional Char- 
acteristic 

IPEDS 

Level 

— 

Institutional Char- 
acteristic 

IPEDS 

Religious Affiliation 

— 

Institutional Char- 
acteristic 

IPEDS 

Tribal College 

— 

Institutional Char- 
acteristic 

IPEDS 

Graduation Rates, Total Cohort (6Years) 

As of 8/31/10 

Student Success 

IPEDS 

Retention Rates, Total Cohort (1 Year) 

Fall 2010 

Student Success 

IPEDS 


SAT Math 75th Percentile Score: /ncludes 
new students admitted the summer 
prior to that fall. 

Completions 

Bachelor's Degrees Conferred: Awards/ 
degrees conferred. 

Enrollment 

Estimated Fall Enrollment: Early estimate 
of enrollment for all levels for full- and 
part-time students. 

Full-Time Equivalent (FTE): The FTE of 
the institution's part-time enrollment 
is estimated and then added to the 
full-time enrollment of the institution. 
The FTE of part-time enrollment is 
estimated by multiplying the part-time 
enrollment by factors that vary by 
control and level of institution and level 
of student. 

Total Enrollment, Unduplicated:The sum 
of students enrolled for credit with 
each student counted only once during 
the reporting period, regardless of 
when the student enrolled. 

Percentage of Classes Enrolling Fewer 
than 20 Students:The percentage of 
undergraduate classes, excluding 
class subsections, with fewer than 20 
students enrolled during fall semester. 

Faculty 

Average Salary Equated to 9-Month 
Contracts of Full-Time Instructional 
Staff-All Ranks: Derived by summing 
the equated 9-month outlays for each 
rank and dividing by the total faculty 
on both 9/10 month and 1 1/12 month 
contracts. 
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Full-Time Primary Instruction Head Count: 
Instructional faculty are instruction/ 
research staff employed full time (as 
defined by the institution) whose 
major regular assignment is instruction, 
including those with released time for 
research. 

Part-Time Primary Instruction Head Count: 
Faculty reported to have a primary 
function of instruction that does not 
exceed 50 percent. 

Percentage of Faculty Holding Terminal 
Degrees: The percentage of full-time 
faculty members with a doctorate or the 
highest degree possible in their field or 
specialty during the academic year. 

Financial 

Endowment (FASB): Endowment 
assets (year-end) per FTE enrollment 
for public and private not-for-profit 
institutions using Financial Accounting 
Standards Board (FASB) standards is 
derived as follows: Endowment assets 
(year-end) divided by 12-month FTE 
enrollment. Endowment assets are 
gross investments of endowment funds, 
term endowment funds, and funds 
functioning as endowment for the 
institution and any of its foundations 
and other affiliated organizations. 
Endowment funds are funds whose 
principal is nonexpendable (true 
endowment) and that are intended 
to be invested to provide earnings for 
institutional use. Term endowment 
funds with the following stipulation 
by the donor: the principal may be 
expended after a stated period or on 
the occurrence of a certain event. Funds 
functioning as endowment (quasi- 
endowment funds) are established by 
the governing board to function like 
an endowment fund but that may be 
totally expended at any time at the 
discretion of the governing board. These 
funds represent nonmandatory transfers 


from the current fund rather than a 
direct addition to the endowment 
fund, as occurs for the true endowment 
categories. 

Instructional Expenses per FTE (FASB): 
Includes all expenses of the colleges, 
schools, departments, and other 
instructional divisions of the institution 
and expenses for departmental 
research and public services that are not 
separately budgeted. Includes general 
academic instruction, occupational and 
vocational instruction, special session 
instruction, community education, 
preparatory and adult basic education, 
and remedial and tutorial instruction 
conducted by the teaching faculty. Also, 
includes expenses for both credit and 
noncredit activities. Excludes expenses 
for academic administration if the 
primary function is administration (e.g., 
academic deans). 

Tuition — Total Price for In-District Students 
Living on Campus: Cost of attendance 
for full-time, first-time degree/certificate 
seeking in-district undergraduate 
students living on campus for the 
academic year. It includes in-district 
tuition and fees, books and supplies, 
on-campus room and board, and other 
on-campus expenses. 

Alumni Giving Rate:The average 
percentage of undergraduate alumni 
(full- or part-time students) who 
donated money to the college or 
university for either current operations 
or capital expenses during the specified 
academic year. Rate is calculated by 
dividing the number of alumni donors 
during a given academic year by the 
number of alumni of record for that 
same year. 

Financial Aid 

Percent of Full-Time Undergraduates 


Receiving Federal Grant Aid: Percent 
of undergraduate students receiving 
grant aid from the federal government. 
Undergraduates are students enrolled 
in a 4- or 5-year bachelor's degree 
program, an associate degree program, 
or a vocational or technical program 
below the baccalaureate. 

Institutional Characteristics 

Carnegie Classification-Basic (Arts & 
Sciences or Diverse Fields): Includes 
institutions where baccalaureate 
degrees represent at least 1 0 percent of 
all undergraduate degrees, institutions 
that award fewer than 50 master's 
degrees or fewer than 20 doctoral 
degrees per year. Excludes special focus 
institutions and tribal colleges. 

Carnegie Classification-Enrollment Size 
& Setting: School sizes are classified by 
very small, small, medium, large. Also 
indicates proportion of students living in 
campus housing. 

Carnegie Classification-Undergraduate 
Profile (Transfer and Full-Time 
Proportions): Used in this case study to 
determine selectivity. 

Geographic Region: \J.S. region school 
where institution is located. 

Level: A classification of whether an 
institution's programs are 4-year or 
higher (4 year), 2-year and less than 
4-year (2 year), or less than 2-year. 
Religious Affiliation: Indicates religious 
affiliation (denomination) for private 
nonprofit institutions that are religiously 
affiliated. 

Tribal College: These institutions, with 
few exceptions, are tribally controlled 
and located on reservations, and are all 
members of the American Indian Higher 
Education Consortium. 
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Appendix C. Siena College Peers and Aspirant Lists 


Near Peers 

Almost Peers 

Institution 

Institution 

Allegheny College 

Augustana College 

Cedarville University 

Birmingham Southern College 

Champlain College 

Calvin College 

College of Saint Benedict 

Carroll College 

Concordia College at Moorhead 

Goucher College 

Cornell College 

Hampshire College 

Gordon College 

Houghton College 

Hartwick College 

Lake Forest College 

Hope College 

Lasell College 

Juniata College 

Linfield College-McMinnville Campus 

Messiah College 

Luther College 

Oglethorpe University 

Muhlenberg College 

Saint Michael's College 

Saint Anselm College 

Stonehill College 

Saint Vincent College 

Susquehanna University 

Southwestern University 

Transylvania University 

St. Olaf College 

Wentworth Institute of Technology 

Trine University 

William Jewell College 

Washington College 

Wofford College 

Westmont College 


Tier 1 Aspirants 

Tier II Aspirants 

Institution 

Institution 

Amherst College 

Barnard College 

Bowdoin College 

Beloit College 

Carleton College 

Bucknell University 

Claremont McKenna College 

Centre College 

Colby College 

Davidson College 

Gustavus Adolphus College 

Denison University 

Hamilton College 

DePauw University 

Kenyon College 

Dickinson College 

Pomona College 

Furman University 

Saint Mary's College 

Grinnell College 

Swarthmore College 

Kalamazoo College 

Vassar College 

Macalester College 

Williams College 

Rhodes College 


Saint John's University 

Skidmore College 

The College of Wooster 

Skidmore College 

Wellesley College 

Whitman College 


Student Success 

Graduation Rates , Total Cohort (6 
Years): The number of students from 
the adjusted conferred bachelor's 
degree-seeking cohort who completed 
a bachelor's degree within 1 50 percent 
of normal time (6 years) divided by the 
adjusted cohort. The adjusted cohort 
is the revised cohort minus exclusions 
as reported by the institution as of 1 50 
percent of normal time (6 years). 

Retention Rates , Total Cohort (1 Year): The 
full-time retention rate is the percent 
of the (fall full-time cohort from the 
prior year minus exclusions from the fall 
full-time cohort) that reenrolled at the 
institution as either full- or part-time in 
the current year. 

Appendix D. Peer Selection 
Websites 

American Association of University 
Professors (AAUP): One of the most 
frequently requested comparisons is 
that of faculty compensation. The AAUP 
provides aggregated information from 
the Faculty Salaries Survey (AAUP, 201 2). 
The provided link is a user-friendly 
interface developed by the Chronicle 
of Higher Education. http://chronicle. 
com/article/faculty-salaries-data- 
201 2/1 31431 #id=1 44050 

Association of Governing Boards (AGB): 
This subscription service provides 
financial metrics primarily sourced from 
IPEDS (AGB, 201 2). Multiple years of data 
are available for over 4,000 institutions. 
http://agb.org/benchmarking-service 

CollegeBoard: Both search and 
comparison capabilities are available 
on bigfuture by the CollegeBoard©: 
Compare Colleges website 
(CollegeBoard, 2012). Information is 
limited, probably due to the fact that 
the primary audience is prospective 
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students and their parents. For 
example, information about faculty 
or instructional expenses is not 
available. Additionally, a maximum of 
only three colleges can be compared 
simultaneously. https://bigfuture. 
collegeboard.org/compare-colleges 

College FactuaEThe website provides a 
rating based on a scorecard of statistics 
of user selected colleges (College 
Factual, 201 3). Ratings are based on 
metrics weighted by the user. Again, 
the intended audience is prospective 
students and their parents. http://www. 
collegefactual.com/ 

Collegelnsight: Hosted by The Institute 
for College Access & Success (TICAS), 
this website provides aggregated 
financial aid information (TICAS, 201 3). 
Although the focus of this application is 
affordability, information on enrollment, 
diversity, and student success is also 
available. The search engine is flexible, 
affording selection on multiple values 
for one or more of the following: sector, 
geographic location, enrollment sizes, 
percent of Pell recipients, and tuition. 
Additionally, several years of data are 
available, http://ticas.org/ 

College Measures: Several key 
institutional indicators are automatically 
aggregated by state and nationally on 
this website (College Measures, 201 3). 
Institutional information is displayed 
as a performance scorecard that must 
be viewed separately for each school. 
http://www.collegemeasures.org/ 

College Miner: This website is unique 
because it reports alumni salary 
information (College Miner, 201 3). 
Simultaneous comparisons can only 
be made for a maximum of three 
institutions. The target audience is 
prospective students and parents. 


Priority of this application is ease of 
use and colorful graphics over data. 
http://collegeminer.com/research/ 
outcomestool.aspx 

College Navigator: Provided by NCES, 
this tool narrows college peers by 
level of award, institutional type, and 
geographic location (NCES, 201 3a). 
Because of the information available 
and the interface, this tool and the IPEDS 
Data Center website described below 
are recommended by the authors. 
http://nces.ed.gov/collegenavigator/ 

College News: As with the CollegeBoard, 
College News has a repository of college 
information named Compare Colleges. 
The online application provides the 
rankings of several publications, 
including the U.S. News & World Report 
(College News, 201 3). Also shown is 
information about enrollment size, 
acceptance rates, and tuition. Although 
visually appealing, the interface is 
somewhat confusing and cumbersome. 
http://www.collegenews.com/ 

College Results Online: Sponsored by 
The Education Trust, the origin of the 
data is the same as for this case study, 
primarily IPEDS (The Education Trust, 

201 2). Institution type and geographic 
location limits the number of peers that 
can be selected at one time. http://www. 
collegeresults.org/search_group.aspx 

IPEDS Data Center: The IPEDS Data 
Center, also supported by NCES, 
provides access to data for multiple 
institutions simultaneously (NCES, 

201 3b). The list of frequently used and 
derived variables makes access to an 
otherwise vast and unwieldy inventory 
of data elements somewhat easier. Early 
released data to IPEDS key holders can 
be obtained by request. Most of the data 
for this case study are from the IPEDS 


Data Center, http://nces.ed.gov/ipeds/ 
datacenter/ 

National Assessment of Service and 
Community Engagement (NASCE): 

NASCE provides comparisons among 
participants regarding student service 
engagement (NASCE, 201 2). Available 
data are derived from a survey of 
student service activities and attitudes. 
http://www.siena.edu/pages/5628.asp 
National Association of College and 
University Business Officers (NACUBO): 
Comparative information sourced from 
several NACUBO surveys is available to 
member institutions (NACUBO, 201 2). 

A peer selection tool is among the site's 
capabilities, http://www.nacubo.org/ 
research/NACUBO_benchmarking_tool. 
html 

U.S. News & World Report: For a fee, 
additional data provided to U.S. News 
& World Report can be downloaded for 
participating institutions (U.S. News & 
World Report, 2011). Rankings aside, 
some of the information that can be 
acquired from U.S. News & World Report 
is not readily available elsewhere. 
Among the data elements unique to the 
U.S. News & World Report ranking are (a) 
awarded financial aid packages, (b) class 
size, and (c) high school GPA of entering 
students, http://premium.usnews.com/ 
best-colleges. 
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